Query-Based Hard-Image Retrieval for Object Detection at Test Time

There is a longstanding interest in capturing the error behaviour of object detectors by finding images where their performance likely to be unsatisfactory. In real-world applications such as autonomous driving, it also crucial characterise potential failures beyond simple requirements detection performance. For example, missed pedestrian close an ego vehicle will generally require closer inspection than car distance. The problem predicting at test time has largely been overlooked literature and conventional approaches based on uncertainty fall short that they are agnostic fine-grained characterisation errors. this work, we propose reformulate "hard" query-based hard image retrieval task, queries specific definitions "hardness", offer intuitive method can solve task for large family queries. Our entirely post-hoc, does not ground-truth annotations, independent choice detector, relies efficient Monte Carlo estimation uses stochastic model place ground-truth. We show experimentally applied successfully wide variety which reliably identify given detector without any labelled data. provide results ranking classification tasks using widely used RetinaNet, Faster-RCNN, Mask-RCNN, Cascade Mask-RCNN detectors. code project available https://github.com/fiveai/hardest.